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EXECUTIVE  SUMMARY 


The  Department  of  Homeland  Security  (DHS)  and  the  United  States  Visitor  and  Immigrant 
Status  Indicator  Technology  (US- VIS  IT)  program  are  migrating  from  a two  print  capture 
process  (left  and  right  index  fingers)  to  a ten-print  slap  fingerprint  capture  process  (all  fingers 
on  both  hands).  The  NIST  Biometrics  Usability  group  performed  a usability  test  to  evaluate 
the  time  required  to  acquire  a 10  print  slap  fingerprint  image.  Users  accustomed  with  leaving 
two  index  fingerprints  at  US-VISIT  may  not  be  familiar  with  the  ten  slap  fingerprint  capture 
process.  Thus,  in  addition  to  measuring  the  time  necessary  to  acquire  the  10  print  slap,  the 
study  also  evaluated  various  methods  of  providing  instructional  information  to  the  users. 

This  report  presents  the  results  of  a study  that  examined  the  10-print  collection  process.  The 
study  was  designed  to  answer  three  questions: 

1.  How  long  does  it  take  to  capture  a 10-print  image? 

2.  What  is  the  impact  of  instructional  mode  (poster,  verbal  or  video)  on  user 
performance? 

3.  What  are  the  frequency  and  nature  of  the  errors  that  occur  in  this  process? 

The  experimental  procedure  may  be  summarized  as  follows.  The  participants  were  300 
adults  recruited  from  a pool  of  10,000  people.  There  were  151  women  and  149  men  ranging 
in  ages  from  18  to  over  65  years.  Each  participant  received  instructions  on  how  to  complete 
the  10-print  collection  process  in  one  of  the  three  formats  — a 76  x 115  cm  poster,  verbal 
instructions  spoken  by  the  test  conductor,  or  a 50  second  soundless  video.  The  instructional 
materials  portrayed  three  steps.  Participants  were  asked  to  present  a left  slap,  a right  slap,  and 
both  thumb  prints  (simultaneously)  according  to  the  instructional  materials.  Participants 
were  randomly  assigned  to  either  a right  or  left  leading  hand  condition  for  one  of  the  three 
instructional  techniques. 

Applying  the  definition  of  usability  from  the  International  Organization  for  Standardization 
(ISO)  we  measured  efficiency,  effectiveness,  and  user  satisfaction.  Efficiency  was  measured 
as  the  time  required  for  a participant  to  complete  the  10-print  collection  process  including 
capturing  a right-slap,  left-slap  and  both  thumb  print  images  according  to  the  instructions 
provided  to  the  participant  by  one  of  the  instructional  methods.  For  each  participant,  the  task 
was  successfully  completed  if  all  three  fingerprint  images  were  captured.  Effectiveness  was 
measured  by  the  number  of  participants  who  were  unable  to  complete  the  task  and  by  the 
number  of  errors  incurred  by  the  participants  who  successfully  completed  the  task.  Following 
collection  of  fingerprints,  participants  were  asked  questions  about  their  satisfaction  with  the 
fingerprint  process  relative  to  the  instructional  modes  provided. 

This  report  describes  four  main  results. 
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1 . Participants  who  received  instructions  via  the  poster  had  the  most  difficulty 
performing  the  fingerprinting  task.  Statistical  analysis  shows  that  participants  who 
received  the  poster  instructions  took  significantly  longer  to  complete  the  10-print 
collection  process  than  participants  in  the  video  or  verbal  instructional  groups.  Fewer 
participants  in  the  poster  condition  were  able  to  complete  the  task  and  made 
significantly  more  errors  than  either  the  video  or  verbal  conditions.  Only  56%  of 
these  participants  were  able  to  successfully  complete  the  fingerprinting  process. 

2.  Participants  who  received  the  verbal  and  video  instructions  performed  equally  well. 
Verbal  instructions  were  rated  highest  in  preference  by  participants  followed  by 
video,  and  poster  was  rated  the  lowest  of  the  three  instructional  modes.  Verbal 
instructions  may  be  difficult  in  an  operational  setting.  Video  instructions  may  be 
presented  to  many  people  waiting  in  line  simultaneously  providing  an  effective 
instructional  delivery  method  by  adding  no  additional  time  for  the  operator  during  the 
collection  process  and  still  providing  optimum  capture  times  and  low-error  incidence. 

3.  Operators  are  critical  to  the  acquisition  process.  It  appears  that  operators  are  able  to 
assist  individuals  to  overcome  the  deficits  of  the  instructional  materials.  With 
operator  assistance  98%  of  the  participants  were  able  to  successfully  complete  the 
fingerprint  process. 

4.  On  average  the  time  required  to  capture  a 10-print  sequence  (a  right  slap,  left  slap  and 
simultaneous  thumbs)  without  operator  assistance  ranges  from  48  to  64  seconds  and 
the  medians  ranged  from  45  to  59  seconds,  and  with  operator  assistance  from  50  to  54 
seconds  (the  medians  ranged  from  45  to  46  seconds)  for  our  biometric  system 
configuration  The  minimum  time  to  successfully  complete  a capture  sequence  of  the 

3 images  in  our  configuration  is  approximately  30  seconds.  This  includes 
approximately  2 1 seconds  for  image  capture  and  9 seconds  of  overhead  for 
initialization  and  other  operational  chores. 
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1.  INTRODUCTION 


The  Department  of  Homeland  Security  (DHS)  and  the  United  States  Visitor  and  Immigrant 
Status  Indicator  Technology  (US-VISIT)  program  are  migrating  from  a two  print  capture 
process  (left  and  right  index  fingers)  to  a ten-print  slap  capture  process  (all  fingers  on  both 
hands).  This  transition  is  based  on  recommendations  from  the  National  Institute  of  Standards 
and  Technology  (NIST)  that  were  made  in  the  joint  report  to  Congress  titled  “Summary  of 
NIST  Patriot  Act  Recommendations”  [NIST  1 ].  The  report  indicated  that  “with  available 
fingerprint  scanning  technology,  the  acquisition  of  10  slap  fingerprints  should  take  only 
slightly  more  time  than  the  acquisition  of  two  flat  fingerprints”  [NIST1], 

US-VISIT  processing  is  currently  based  on  a two-print  capture  process,  the  left  and  right 
index  fingers  of  the  individual.  This  is  an  operator  assisted  capture  process  with  an 
experienced  Customs  and  Border  Protection  (CBP)  agent  guiding  each  step  of  processing 
primarily  using  verbal  instructions.  In  addition  to  assistance  from  the  operator,  some  visual 
assistance  is  provided  via  placards  that  have  symbols  describing  the  fingerprint  process  as 
seen  in  Figure  1. 


Figure  1:  US-VISIT  Scanner  Instructional  Placards  (left)  and  Instructional  Handout( right) 


The  US-VISIT  exit  procedure  involves  an  unmanned  kiosk  that  guides  the  user  by  written 
instructions  as  well  as  video/animation.  In  the  exit  procedure,  digital  fingerprints  and 
photographs  are  again  collected  from  the  individual. 
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Figure  2:  US-VISIT  Exit  kiosk 


The  NIST  Biometrics  Usability  group  performed  a usability  test  to  evaluate  the  time  required 
to  acquire  a 10  print  slap  image  h Users  accustomed  with  leaving  two  index  fingerprints  at 
US-VISIT  may  not  be  familiar  with  the  ten  slap  fingerprint  capture  process.  In  addition  to 
measuring  the  time  necessary  to  acquire  the  10  slap  image,  the  study  also  evaluated  how  to 
present  instructional  information  to  the  users.  The  study  was  designed  to  answer  three 


questions: 


1 . How  does  the  instructional  mode  (poster,  video,  or  verbal)  affect  user  performance? 

2.  How  long  does  it  take  to  capture  a 1 0-print  image? 

3.  What  are  the  frequency  and  nature  of  the  errors  that  occur  in  this  process? 

2.  METHOD 

2.1  PARTICIPANTS 

The  participants  were  300  adults  recruited  from  a pool  of  10,000  people  who  had  previously 
agreed  to  participate  in  usability  tests.  There  were  151  women  and  149  men  ranging  in  ages 
from  18  to  over  65  years. 


1 These  tests  were  performed  for  the  Department  of  Homeland  Security  in  accordance  with  section  303  of  the 
Border  Security  Act,  codified  as  8 U.S.C.  1732.  Specific  hardware  and  software  products  identified  in  this 
report  were  used  in  order  to  perform  the  evaluations  described  in  this  document.  In  no  case  does  such 
identification  imply  recommendation  or  endorsement  by  the  National  Institute  of  Standards  and  Technology, 
nor  does  it  imply  that  the  products  and  equipment  identified  are  necessarily  the  best  available  for  the  purpose. 
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Figure  3:  Participant  Age  Range 

Of  the  300  participants,  265  were  right  handed,  31  left  handed,  two  ambidextrous,  and  two 
did  not  respond.  As  representative  of  the  general  population  most  of  the  participants  were 
right-handed  at  89%,  while  1 1%  were  either  left-handed  or  ambidextrous.  Available 
research  shows  that  approximately  87%  of  the  general  population  is  right  handed  with  the 
remaining  13%  of  the  population  either  left  handed  or  ambidextrous. [PORAC] 


Figure  4:  Participant  Handedness 

The  participants  ranged  in  self-reported  height  from  56  inches  (142  cm)  to  79  inches  (201 
cm)  with  the  range  being  fairly  normally  distributed  with  an  average  height  of  70.2  inches 
( 1 78.3  cm)  for  males  and  64.4  inches  ( 1 63.6)  for  females.  According  to  the  Centers  for 
Disease  Control  and  Prevention  (CDC)  the  mean  individual  height  of  men  is  69  inches  (175 
cm)  and  the  mean  individual  height  of  women  is  63  inches  (160  cm)  in  the  US[CDC].  The 
height  data  collected  showed  that  our  mean  population  height  is  within  2%  of  the  mean  in  the 
US  general  population.  According  to  the  World  Health  Organization  the  worldwide  mean 
individual  male  height  is  5 feet  8 inches  (173  cm)  and  the  female  height  is  5 feet  2 inches 
(158  cm). 
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Figure  5:  Participant  Height 


No  participants  worked  in  law  enforcement  or  with  fingerprints.  All  participants  had  self 
reported,  normal  color  vision,  and  all  ten  digits.  All  participant  spoke  English  fluently. 

The  participants  indicated  whether  they  had  been  previously  fingerprinted. 

Of  the  300  participants  70%  had  been  fingerprinted,  only  9.7%  had  been  electronically 
fingerprinted  (see  Figure  6).  This  is  consistent  with  a telephone  survey  on  public  attitudes 
towards  biometrics  that  was  conducted  in  September  2001  where  69%  had  been  fingerprinted 
and  August  2002  where  66%  had  been  fingerprinted  [ORC]. 


Experience  With  Being  Fingerprinted 


Figure  6:  Previous  Fingerprint  Experience 

Participants  were  also  asked  “How  concerned  were  you  about  having  your  fingerprints 
recorded?”  - Very  concerned.  Fairly  concerned,  Not  very  concerned.  Not  at  all  concerned,  or 
Don’t  know.  Of  the  300  participants  84%  were  not  at  all  or  not  very  concerned  about  having 
their  fingerprints  recorded  (see  Figure  7).  This  is  consistent  with  previous  surveys  that 
suggests  that  people  have  positive  attitudes  towards  finger-imaging  and  are  comfortable  with 
biometric  techniques  [ORC], 
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□ Very 


■ Don't  Know 


Figure  7:  Survey  Question  “How  concerned  were  you  about  having  your  fingerprints  recorded?” 

Finally  participants  were  asked  if  they  would  be  in  favor  or  opposed  to  providing  fingerprint 
images  as  a means  of  establishing  identity  for  passport  purposes.  Of  the  300  participants  230 
or  77%  were  in  favor  or  strongly  in  favor.  This  study’s  population  mirrored  the  high 
acceptance  for  law  enforcement  to  require  fingerprint  scans  to  verify  identity  for  passports 
of  the  previous  survey  of  88%  [ORC], 

2.2  MATERIALS 

Three  different  types  of  instructional  materials  were  used.  The  participants  were  given 
instructions  on  how  to  give  digital  fingerprints  via  verbal  instructions  spoken  by  the  test 
conductor  (see  Appendix  D),  a 50  second  soundless  video  presented  on  a 17”  monitor,  or  a 
76  x 1 15  cm  poster  (see  Appendix  C).  The  instructional  materials  portrayed  three  steps: 

1 . Presenting  four  fingers  of  the  right  hand  (right  slap), 

2.  Presenting  four  fingers  of  the  left  hand  (left  slap),  and 

3.  Presenting  both  thumbs  to  the  fingerprint  sensor. 

The  start  of  the  right  slap  step  was  indicated  by  the  rightmost,  of  three,  green  Light  Emitting 
Diodes  (LEDs)  illuminating.  The  start  of  the  left  slap  step  was  indicated  by  the  leftmost 
LED  illuminating.  The  start  of  the  thumbs  step  was  indicated  by  the  center  LED 
illuminating.  Each  step  ended  when  their  respective  LED  darkened. 

The  three-slap  capture  process  where  thumbs  are  captured  simultaneously  as  opposed  to  a 
four-slap  capture  process  where  thumbs  are  captured  individually  was  selected  because  of  the 
predominant  use  of  the  three-slap  process  by  law  enforcement.  One  of  the  reason  for  this 
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popularity  is  because  capturing  both  thumbs  together  can  virtually  eliminate  the  possibility  of 
the  user  transposing  the  left  and  right  thumb  prints  during  the  capture  process. 


1.  2. 


Figure  8:  Fingerprint  Process  Poster  (Right  Hand  Start) 


There  were  two  versions  of  each  instructional  material.  One  version  started  with  a right  hand 
and  a second  version  started  with  a left  hand. 

2.3  EQUIPMENT 

2.3.1  Digital  Fingerprint  Scanner 

The  digital  fingerprint  scanner  selected  for  this  experiment  was  the  Smiths-Heimann 
(Crossmatch)  LS2  Check  digital  fingerprint  scanner  [SHB]  (see  Figure  9) . The  LS2  scanner 
has  a large  3.2  inch  x 2.9  inch  platen  (81mm  x 74mm)  that  allows  for  the  capture  of  one  or 
more  fingers  simultaneously.  The  LS2  scanner  has  a height  of  6 inches  (152mm).  There  are 
three  Light  Emitting  Diode  (LED)  indicators  on  the  top  surface  of  the  scanner.  These 
indicators  are  capable  of  red  and  green  illumination.  The  scanner  was  situated  on  a work 
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surface  height  of  36  inches  (914  mm)  which  has  been  identified  as  an  optimal  height  for  the 
collection  of  ten-print  slap  images  [NIST2]. 


Figure  9:  Fingerprint  Scanner 


The  fingerprint  scanner’s  top  was  covered  by  a piece  of  opaque  paper  to  obscure  any 
markings.  The  paper  had  openings  cut  into  it  to  allow  manual  access  to  the  scanner’s  platen 
and  visual  access  to  the  three  LED  indicators  (see  Figure  10). 
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Figure  10:  Fingerprint  Scanner  (Covered) 


2.3.2  Capture  Software 

This  experiment  utilized  the  MB  ARK  software  package  [NIST5]  for  controlled  capture  of 
images  from  a given  participant.  The  custom  software’s  high-level  operational  flow  can  be 
described  in  the  following  three  steps: 

1 ) Subject  is  prompted  for  a hand/finger  combination 

2)  Once  the  hand/finger  is  detected,  capture  images  for  a set  amount  of  time 

3)  Repeat  step  2 for  the  next  finger/hand  combination  until  all  combinations  are 
complete 

2.3.3  Behavioral  Observation  Software 

Noldus  Observer  v5.0.31  running  on  Desktop  PC  and  Noldus  Pocket  Observer  v2.0 
(handheld  version)  running  on  HP  Ipaq  2495  PDAs  were  used  to  record  errors  and  the  timing 
of  events. 

2.4  FACILITY 

The  test  was  conducted  in  a usability  testing  facility  consisting  of  two  rooms  separated  by  a 
one-way  mirror.  The  participant,  test  conductor  and  scanner  operator  were  in  one  room  and 
an  observer  was  in  the  other  room  (see  Figure  11). 
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Figure  11:  Test  Room  Layout 


2.5  PROCEDURE 

Each  participant  had  three  steps  to  complete.  Participants  were  asked  to  present  a left  slap,  a 
right  slap,  and  both  thumb  prints  (simultaneously).  Participants  were  randomly  assigned  to 
either  a right  or  left  leading  hand  condition  for  one  of  the  three  instructional  techniques. 


Instructions 

Participants 

Leading  Hand 

Right 

Left 

Verbal 

100 

50 

50 

Poster 

100 

50 

50 

Video 

100 

50 

50 

Table  1:10  Print  Experimental  Design 


Every  participant  was  greeted  and  thanked  for  their  participation.  After  a brief  description  of 
the  fingerprint  scanner,  a consent  and  demographic  form  was  filled  out  (see  Appendix  A). 
The  participant  was  then  given  instructions  on  how  to  present  their  fingerprints  by  one  of 
three  techniques.  In  one  condition  the  instructions  were  given  verbally  by  the  test  conductor 
(see  Appendix  D for  text).  In  the  second  condition  the  participant  was  shown  a poster 
depicting  the  three  presentations  (right  slap,  left  slap  and  thumbs).  Each  depiction  of  a 
presentation  consisted  of  a picture  for  each  of  four  stages  (see  Appendix  C).  The  first  picture 
showed  the  scanner  with  no  LEDs  illuminated.  The  second  picture  showed  the  scanner  with 
an  illuminated  LED  and  the  corresponding  hand  over  the  scanner  platen.  The  third  picture 
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showed  either  fingers  or  thumbs  pressing  on  the  platen  while  the  LED  was  illuminated.  The 
fourth  picture  showed  a non-illuminated  scanner  with  a hand  above  the  platen  and  a arrow, 
pointing  upwards,  over  the  hand.  The  third  presentation  technique  consisted  of  a 50  second, 
silent,  live  action  video  of  the  three  presentations.  The  video  was  framed  so  only  the 
scanner  and  the  hand  was  visible  (see  Figure  12).  The  participant  pool  for  each  instructional 
method  was  divided  equally  so  half  of  the  participants  would  receive  a instruction  that  started 
with  the  right  hand  and  the  other  half  would  receive  instructions  that  started  with  the  left 
hand. 


Figure  12:  Capture  of  Instructional  Video 


Participants  could  receive  the  instructions  as  many  times  as  they  wanted  and  were  asked  if 
they  had  any  questions  after  receiving  instructions.  Participants  were  instructed  to  walk  over 
to  the  scanner  area  once  they  were  satisfied  with  the  instructions.  The  test  conductor  stopped 
the  video  or  turned  the  poster  away  from  the  participant  so  neither  could  be  seen  from  the 
fingerprint  capture  area.  Once  the  participant  was  standing  in  front  of  the  scanner,  the 
operator  started  the  scanner  and  indicated  to  the  participant  that  they  may  begin.  Depending 
on  the  experimental  condition  either  the  rightmost  or  leftmost  LED  on  the  scanner  would 
illuminate.  The  scanning  system  would  give  the  participant  three  opportunities  to  present  for 
each  step.  Once  a capture  sequence  was  started  the  system  would  wait  25  seconds  to  sense  if 
fingers  were  placed  on  the  platen.  If  the  system  sensed  fingers  within  the  25  seconds  it 
would  capture  images  for  five  seconds.  At  five  seconds  the  system  would  check  to  see  if  it 
still  sensed  a hand.  If  it  did  sense  a hand,  the  step  was  marked  as  successful,  if  not,  an  error 
was  generated  and  the  scanner  would  offer  any  remaining  opportunities.  If  the  system  did 
not  sense  fingers  on  the  platen  within  the  25  seconds,  it  would  re-initiate  the  capture 
sequence  up  to  three  times.  If  nothing  was  sensed  in  three  opportunities,  the  system  would 
mark  that  step  as  unsuccessful  and  move  on  to  the  next  step,  as  indicated  by  a new  LED 
illuminating.  A visual  representation  of  this  sequence  is  provided  in  Figure  13. 
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Using  the  Noldus  systems,  the  test  conductor  and  observer  marked  certain  participant 
actions  which  the  systems  would  timestamp  (see  Appendix  E).  The  scanner  software  also 
created  a time-stamped  log  of  events. 

After  the  completion  of  the  fingerprint  scanning  the  participant  was  asked  to  fill  out  a 
questionnaire  (see  Appendix  B). 
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3.  RESULTS 

3.1  USABILITY  METRICS 

According  to  ISO  9241-1 1 [ISO],  usability  is  defined  as  “the  extent  to  which  a product  can 
be  used  by  specified  users  to  achieve  specified  goals  with  effectiveness,  efficiency  and 
satisfaction  in  a specified  context  of  use”.  The  standard  identifies  three  areas  of 
measurement:  effectiveness,  efficiency,  and  user  satisfaction,  where 

• efficiency  is  a measure  of  the  resources  expended  in  relation  to  the  accuracy  and 
completeness  with  which  users  achieve  goals.  Efficiency  is  related  to  productivity 
and  is  generally  measured  as  task  time 

• effectiveness  is  a measure  of  the  accuracy  and  completeness  with  which  users  achieve 
specified  goals.  Common  metrics  include  completion  rate  and  number  of  errors. 

• user  satisfaction  is  the  degree  to  which  the  product  meets  the  users’  expectations — a 
subjective  response  in  terms  of  ease  of  use,  satisfaction,  and  usefulness. 

In  this  study,  we  used  the  ISO  definition  and  measured  efficiency,  effectiveness,  and  user 
satisfaction. 

3.1.1  Efficiency 

We  measured  efficiency  as  the  time  required  for  a participant  to  complete  the  task.  A task  is 
defined  as  capturing  a right-slap,  left-slap  and  both  thumb  prints  images  according  to  the 
instructions  provided  to  the  participant  by  one  of  the  instructional  methods.  For  each 
participant,  the  task  was  successfully  completed  if  fingerprint  images  were  captured  for  the 
three  slaps 

As  described  in  the  procedure  section,  the  leading  hand  of  the  instruction  sequence  was 
counterbalanced.  While  it  was  determined  later  in  analysis  that  there  was  no  interaction 
between  leading  hand  and  any  other  factor. 
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Due  to  hardware  and  software  failures  during  the  data  collection  process,  timing  data  for 
several  participants  was  found  to  be  corrupt.  Data  was  available  for  284  participants  of  the 
300  participants.  Table  2 indicates  the  number  of  participants  for  each  instructional  method 
where  data  was  completely  available  as  well  as  the  breakdown  of  leading  hand  start  sequence 
provided  by  the  instructional  material  to  each  group. 


Instructional 

Method 

Number  of 
Participants 

Instructions  Start 
With  Right  Hand 

Instructions  Start 
With  Left  Hand 

Poster 

96 

46 

50 

Verbal 

90 

51 

39 

Video 

98 

50 

48 

Table  2:  Available  data  after  elimination  of  corrupted  trials 


This  subset  of  the  population  was  further  refined  for  this  discussion  by  those  participants 
where  the  three  images  were  collected  thus  successfully  completing  the  task.  Of  the  284 
participants  where  data  was  available,  222  or  78%  successfully  completed  the  task.  Table  3 
indicates  the  number  of  participants  for  each  instructional  method  that  successfully 
completed  the  task. 


Instructional  Method 

Number  of  Participants 

Poster 

52 

Verbal 

85 

Video 

85 

Table  3:  Participants  who  successfully  completed  the  task 


For  completeness  we  identified  a sequence  of  events  that  are  required  to  complete  the  overall 
task  from  receiving  the  instructions  to  successful  capture  of  fingerprint  images.  In  addition 
to  capturing  the  time  required  to  complete  the  overall  task  we  also  measured  the  time 
required  to  complete  these  individual  events  or  subtasks.  These  events  are  illustrated  in 
Figure  14.  Individual  events  include  the  presentation  of  the  instructions,  the  approach  to  the 
sensor,  and  capturing  the  images. 
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Figure  14:  Event  Timeline  for  Instructional  Modes 


Presentation  of  Instructional  Material 

The  first  event  was  the  presentation  of  the  instructional  material.  We  measured  this  as  the 
time  the  participant  entered  the  room  until  he/she  crossed  a taped  line  on  the  floor.  Once  the 
participant  crossed  this  line  (Figure  1 1),  he/she  had  clearly  finished  viewing  the  instructional 
materials.  The  mean  and  median  times  that  participants  required  to  receive  each  respective 
instructional  method  is  provided  in  Table  4.  The  minimum  time  to  receive  the  verbal 
instructions  was  approximately  45  seconds.  The  minimum  time  to  view  the  video  was  50 
seconds.  The  minimum  length  of  time  the  participants  viewed  the  poster  was  at  their 
discretion. 
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Instructional  Method 

Mean  Time  (sec) 

Median  Time  (sec) 

Poster 

31.27 

29.60 

Verbal 

65.34 

58.85 

Video 

86.93 

81.15 

Table  4:  Time  to  receive  instructional  materials 


A histogram  with  a normal  probability  density  plot  of  task  times  showed  that  the  data  did  not 
follow  a normal  distribution.  This  was  verified  by  the  Shapiro  test  for  normality.  All  tests 
give  p-values  <0.01.  It  is  clear  these  variable  are  definitely  not  normally  distributed.  Other 
tests  such  as  the  Chi-squared  test  and  the  tests  based  on  skewness  and  kurtosis  were  also 
applied  again,  all  give  p-value  <0.01. 

Due  to  the  non-normal  distribution  of  the  data  the  Kruskal- Wallis  test  and  the  median  test 
were  used  to  determine  if  the  medians  for  the  3 methods  are  different.  Both  tests  give  a p- 
value  < 0.01  indicating  that  the  medians  for  the  3 methods  are  different.  To  complete  the 
analysis,  the  Mann-Whitney  test  was  used  to  test  the  pairs. 

1 . For  Method  1 vs.  Method  2 (poster  vs.  verbal),  the  medians  of  the  two  methods  are 
statistically  different,  p-value  < 0.01 . 

2.  For  Method  1 vs.  Method  3 (poster  vs.  video),  the  medians  of  the  two  methods  are 
statistically  different,  p-value  <0.01. 

3.  For  Method  2 vs.  Method  3 (verbal  vs.  video),  the  medians  of  the  two  methods  are 
statistically  different,  p-value  <0.01. 

Therefore,  participants  spent  the  least  amount  of  time  with  the  poster  instructions,  followed 
by  the  verbal  instructions.  Participants  spent  the  most  time  with  the  video  instructions  . 

Approaching  the  Sensor  and  Capturing  Images 

The  second  event  was  the  approach  to  the  sensor.  This  task  time  was  measured  from  the 
time  the  participant  crossed  the  taped  line  on  the  floor  until  the  software  application  end.  The 
mean  and  median  time  taken  are  presented  in  Table  5. 


Instructional  Method 

Number  of  Participants 

Mean  Time 
(sec) 

Median  Time 
(sec) 

Poster 

52 

114.40 

91.50 

Verbal 

85 

86.40 

65.00 

Video 

85 

76.30 

65.00 

Table  5:  Time  from  approaching  sensor  to  end  of  capture 
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Due  to  the  non-normal  distribution  of  the  data  the  Kruskal-Wallis  test  and  the  median  test 
were  used  to  determine  if  the  medians  for  the  three  methods  are  different.  Both  tests  give  a 
p-value  <0.01  indicating  that  the  medians  for  the  three  methods  are  different.  To  complete 
the  analysis,  the  Mann- Whitney  test  was  used  to  test  different  medians  for  Methods  2 and  3 
(verbal  and  video)  which  resulted  in  a p-value  > 0.05.  Thus  they  are  not  different.  The  time 
to  complete  the  tasks  when  given  the  poster  is  statistically  significant  from  both  the  time  to 
complete  the  task  when  receiving  verbal  and  video  instructions,  but  there  is  no  significant 
difference  between  the  task  completion  time  for  the  video  or  verbal  instructions. 


Capturing  Images 

The  third  event  was  capturing  the  images.  This  time  is  measured  from  the  software 
application  capture  start  to  the  application  capture  end.  The  minimum  time  to  successfully 
complete  a capture  sequence  is  approximately  30  seconds.  This  includes  approximately  2 1 
seconds  for  image  capture  and  9 seconds  of  overhead  for  initialization  and  other  operational 
chores.  The  mean  and  median  times  are  presented  in  Table  6. 


Method 

Participants 

Mean  (sec) 

Median  (sec) 

Poster 

52 

64.13 

58.50 

Verbal 

85 

48.21 

45.00 

Video 

85 

50.87 

46.00 

Table  6:  Time  of  software  start  capture  and  end  capture 


The  Kruskal-Wallis  test  and  the  median  test  were  used  to  determine  if  the  medians  for  the  3 
methods  are  different.  Both  tests  give  a p-value<0.01  indicating  that  the  medians  for  the  3 
methods  are  different.  To  complete  the  analysis,  the  Mann- Whitney  test  was  used  to  test 
different  medians  for  Methods  2 and  3 (verbal  and  video)  which  resulted  in  a p-value  > 0.05. 
Thus  they  are  not  different.  Therefore,  participants  in  the  poster  condition  took  significantly 
longer  than  either  video  or  verbal  conditions  to  complete  the  tasks. 

Total  Time 

Finally,  we  measured  the  total  time  a participant  took  to  complete  the  task,  from  receiving 
the  instructions  to  end  of  capture.  The  mean  and  median  times  are  presented  in  Table  7. 


Method 

Participants 

Mean  (sec) 

Median  (sec) 

Poster 

52 

145.67 

126.85 

Verbal 

85 

151.73 

129.32 

Video 

85 

163.22 

152.18 

Table  7:  Total  Time 
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Again,  due  to  the  non-normal  distribution  of  the  data,  the  Kruskal- Wallis  test  and  the  median 
test  were  used  to  determine  if  the  medians  for  the  3 methods  are  different.  Both  tests  give  a 
/7-value<0.01  indicating  that  the  medians  for  the  3 methods  are  different.  To  complete  the 
analysis,  the  Mann-Whitney  test  was  used  to  test  the  pairs. 

1.  For  Method  1 vs.  Method  2 (poster  vs.  verbal),  the  medians  of  the  two  methods  are 
not  statistically  different,  p-value  > 0.05. 

2.  For  Method  1 vs.  Method  3 (poster  vs.  video),  the  medians  of  the  two  methods  are 
statistically  different,  p-value  <0.01 . 

3.  For  Method  2 vs.  Method  3 (verbal  vs.  video),  the  medians  of  the  two  methods  are 
statistically  different,  p-value  < 0.01 . 

Therefore,  the  difference  of  the  methods  is  based  on  the  significant  difference  between  poster 
and  video  and  verbal  and  video. 

Error-free  Image  Capture 

In  order  to  eliminate  the  effect  of  errors  on  the  time  required  to  capture  images,  we  identified 
those  participants  who  successfully  completed  the  task  with  no  errors.  Of  the  222 
participants  who  successfully  completed  the  task,  155  completed  the  task  with  no  errors. 
Again,  we  measured  the  time  to  capture  the  images  from  the  software  application  capture 
start  to  the  application  capture  end.  The  minimum  time  to  successfully  complete  a capture 
sequence  is  approximately  30  seconds.  This  includes  approximately  21  seconds  for  image 
capture  and  9 seconds  of  overhead  for  initialization  and  other  operational  chores.  The  mean 
and  median  times  are  presented  in  Table  8. 


Method 

Participants 

Mean  (sec) 

Median  (sec) 

Poster 

17 

59.20 

55.00 

Verbal 

68 

46.10 

45.00 

Video 

70 

47.60 

44.00 

Table  8:  Time  of  software  start  capture  and  end  capture  with  no  errors 


Due  to  the  non-normal  distribution  of  the  data,  the  Kruskal-Wallis  test  and  the  median  test 
were  used  to  determine  if  the  medians  for  the  3 methods  are  different.  Both  tests  give  a p- 
value  <0.01  indicating  that  the  medians  for  the  3 methods  are  different.  To  complete  the 
analysis,  the  Mann-Whitney  test  was  used  to  test  the  pairs. 

1.  For  Method  1 vs.  Method  2 (poster  vs.  verbal),  the  medians  of  the  two  methods  are 
statistically  different,  p < 0.01. 

2.  For  Method  2 vs.  Method  3 (verbal  vs.  video),  the  medians  of  the  two  methods  are 
not  statistically  different,  p-value  > 0.05. 
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3.  For  Method  1 vs.  Method  3 (poster  vs.  video),  the  medians  of  the  two  methods  are 
statistically  different,  p < 0.01. 

Therefore,  the  difference  of  the  methods  is  based  on  the  significant  difference  between  poster 
and  video  and  verbal  and  video. 

3.1.2  Effectiveness 

In  this  experiment  we  measured  effectiveness  in  two  ways: 

1 . the  number  of  participants  who  were  unable  to  complete  the  task 

2.  the  number  of  errors  incurred  by  the  participants  who  successfully  completed  the 
task. 

Of  the  284  participants  for  whom  data  was  available,  78%  were  able  to  complete  the  task. 

The  remaining  62  were  unable  to  complete  the  task,  meaning  that  they  were  unable  to  leave  a 
left  slap,  right  slap,  and  both  thumb  images.  For  each  image  these  participants  were  given 
three  attempts.  Table  9 shows  the  number  of  participants  that  were  unable  to  complete  the 
task  by  instructional  mode.  The  overwhelming  majority  (71%)  of  participants  who  were 
unable  to  complete  the  task  were  in  the  poster  condition. 


Instructional  Method 

Number  of  Participants 

Poster 

44 

Verbal 

5 

Video 

13 

Table  9:  Subjects  that  did  not  successfully  complete  the  task 


Next  we  measured  the  number  of  errors  incurred  by  the  participant  for  each  instructional 
mode  for  those  that  successfully  completed  the  task.  Thus  the  participant  did  not  conform  to 
the  instructions  but  the  attempt  was  acceptable  (prints  were  captured)[NIST4]  to  complete 
the  task  within  three  tries  for  each  stage.  Errors  included  placing  wrong  hand  on  the  scanner, 
lifting  hand  too  soon,  placing  hand  down  too  late,  and  a general  error  such  as  not  placing  a 
hand  down  at  all.  The  following  table  shows  the  mean  and  median  number  of  errors  incurred. 
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For  the  222  participants  that  successfully  completed  the  task,  the  Table  10  shows  the  number 
of  emors  incurred. 


Instructional 

Method 

Number  of 
Participants 

Mean  Error 
Count 

Median  Error 
Count 

Max  Error 
Count 

Poster 

52 

2.02 

2 

8 

Verbal 

85 

0.29 

0 

3 

Video 

85 

0.33 

0 

3 

Table  10:  Error  Summary  For  Completed  Cases 


For  eiTor  count,  the  Kruskal-Wallis  test  and  the  median  test  both  give  p-value  < 0.01 
indicating  that  the  medians  for  the  3 methods  are  different.  The  Mann-Whitney  test  was  used 
to  test  different  medians  for  Methods  2 and  3 with  p-value=0.82.  Thus  they  are  not  different. 

Finally,  of  the  222  that  successfully  completed  the  task  155  or  55%  completed  error  free 
trials.  Table  1 1 shows  the  distribution  by  instructional  mode.  Fewer  participants  from  the 
poster  condition  had  error  free  tasks  than  either  the  verbal  or  video  conditions. 


Method 

Error  Free 

% of  Successful 

Poster 

17 

33% 

Verbal 

68 

80% 

Video 

70 

82% 

Table  11:  Error  Free  Completion  by  Instructional  Method 

In  conclusion,  fewer  participants  in  the  poster  condition  were  able  to  complete  the  task  and 
made  significantly  more  errors  than  either  the  video  or  verbal  conditions.  There  is  no 
significant  difference  between  the  error  count  for  the  video  or  verbal  instructions. 


3.1.3  User  Satisfaction 

Each  participant  was  given  a satisfaction  survey  after  completing  the  test  (See  Appendix  B 
for  the  actual  questionnaire).  The  questions  focused  on  the  fingerprint  process  and  the 
specific  instructional  mode  provided. 

The  following  7 questions  rating  the  instructional  materials  on  a 5-point  Likert-scale 
[LIKERT]  where  1 was  poor  and  5 was  good  were  answered  by  each  participant: 

1.  The  information  provided  was  effective  in  helping  me  complete  the 
fingerprint  process. 

2.  The  information  provided  clearly  described  the  fingerprint  process. 
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3.  The  information  provided  was  confusing. 

4.  It  was  clear  which  hand  was  to  be  scanned  first. 

5.  The  order  of  fingerprint  capture  was  clear. 

6.  It  was  clear  how  to  position  your  hand  on  the  sensor 

7.  It  was  clear  when  to  remove  your  hand. 

Overall  participants  were  satisfied  with  the  instructions  they  were  provided  to  complete  the 
fingerprint  process.  The  verbal  instructions  were  rated  highest  by  the  participants  followed 
by  the  video  and  the  poster  was  rated  the  lowest  of  the  three  instructional  modes.  The  verbal 
instructions  were  ranked  the  highest  on  all  but  one  question.  Participants  ranked  the  video  as 
clearer  with  respect  to  how  to  position  your  hand  on  the  sensor  than  either  the  poster  or  the 
verbal  instructions.  Figure  15  through  Figure  20  indicate  the  results  of  the  7 questions 
concerning  the  effectiveness  and  clarity  of  the  instructional  materials. 


1 2 3 4 5 


□ Roster 

□ Video 
■ Verbal 


Rating  Scale  (1=Poor,  5=Good) 


Figure  15:  Survey  Question  “The  information  provided  was  effective  in  helping  me  complete  the  fingerprint 

process.” 
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90 
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1 2 3 4 5 

Rating  Scale  (1=Poor,  5=Good) 


Figure  16:  Survey  Question  “The  information  provided  was  confusing.” 
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□ Fbster 

□ Video 
■ Verbal 


Rating  Scale  (1=Poor,  5=Good) 


Figure  17:  Survey  Question  “It  was  clear  which  hand  was  to  be  scanned  first.” 


1 2 3 4 5 


□ Fbster 

□ Video 
■ Verbal 


Rating  Scale  (1=Poor,  5=Good) 


Figure  18:  Survey  Question  “It  was  clear  how  to  position  your  hand  on  the  sensor.” 


Rating  Scale  (1=Poor,  5=Good) 


Figure  19:  Survey  Question  “The  order  of  fingerprint  capture  was  clear.” 
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100 


90 


□ Raster 

□ Video 
■ Verbal 


Rating  Scale  (1=Poor,  5=Good) 


Figure  20:  Survey  Question  “It  was  clear  when  to  remove  your  hand.” 

After  rating  the  instructional  materials,  the  participants  were  asked  two  open-ended 
questions.  First  they  were  asked  “What  was  the  most  confusing  part  of  the  process?” 
followed  by  “What  additional  information  would  you  recommend  to  improve  the  fingerprint 
capture  process?”. 

Most  of  the  comments  from  the  participants  that  viewed  the  poster  revolved  around  when  to 
remove  their  hand  from  the  platen.  Of  the  100  participants  that  received  the  poster  as  the 
instructional  material  39%  commented  that  they  were  unsure  when  to  remove  their  hand 
from  the  scanner.  In  addition,  20%  commented  that  they  missed  or  did  not  understand  the 
lights.  Fourteen  percent  of  the  participants  mentioned  that  they  were  confused  about  how 
long  to  leave  their  hand  on  the  scanner.  Only  7%  commented  on  how  to  position  their  hand 
on  the  scanner.  Many  (37%)  suggested  that  adding  text  to  the  poster  would  be  helpful. 
Seventeen  percent  responded  that  the  green  blinking  light  should  be  emphasized  in  order  to 
indicate  when  to  remove  your  hand,  10%  recommended  that  the  poster  should  include 
additional  timing  information. 

Those  who  saw  the  video  also  commented  that  they  were  unsure  of  when  to  remove  their 
hand  from  the  scanner,  but  this  time  only  18%  mentioned  this,  1 1%  commented  that  they 
were  confused  by  the  lights,  and  9%  were  unsure  how  long  to  leave  their  hand  on  the 
scanner.  However,  14%  mentioned  that  they  were  unsure  how  to  position  their  hands  on  the 
scanner.  Just  as  in  the  poster,  many  45%  suggested  that  the  video  should  include  text  or 
audio.  Another  20%  indicated  that  the  video  or  the  scanner  should  emphasize  how  to 
position  your  hand.  Only  7%  felt  the  video  needed  to  include  timing  information. 

Most  participants  stated  that  the  verbal  instructions  clearly  described  the  tasks,  32%  had  no 
comments  or  complaints  and  22%  had  favorable  comments.  Concern  about  how  to  position 
their  hand  on  the  scanner  was  mentioned  by  20%  of  the  verbal  participants.  Thus  adding 
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instructions  on  how  to  position  your  hand  was  suggested  by  22%  of  the  participants  while 
17%  would  include  a visual  element  with  the  verbal  instructions. 

Finally,  cleanliness  of  the  scanner  was  an  issue  that  several  participants  identified.  About 
2%  of  the  participants  did  express  concern  about  the  cleanliness  of  the  devices  with  which 
they  would  have  physical  contact.  However,  when  participants  were  questioned  about  their 
thoughts  on  cleanliness,  they  indicated  that  they  were  not  as  concerned  in  this  particular 
environment  since  it  was  a controlled  laboratory  setting.  They  assumed  the  platen  was 
cleaned  routinely  and  adequately.  But  they  expressed  concerns  about  the  operational  setting, 
indicating  that  they  would  immediately  wash  their  hands  or  use  a wet  wipe  after  touching  the 
fingerprint  scanner  [NIST3]. 

3.1.4  Reference  Task 

After  completing  the  user  satisfaction  survey,  participants  were  asked  to  repeat  the 
fingerprint  collection  task.  This  second  set  of  images  is  important  in  order  to  perform 
matches  of  images.  This  fingerprint  task  was  operated  assisted.  Once  again,  we  collected 
timing  and  error  data.  Time  is  measured  from  the  software  application  capture  start  to  the 
application  capture  end.  The  minimum  time  to  successfully  complete  a capture  sequence  is 
approximately  30  seconds.  This  includes  approximately  21  seconds  for  image  capture  and  9 
seconds  of  overhead  for  initialization  and  other  operational  chores. 

Of  the  276  participants  who  agreed  to  repeat  the  collection  process  all  but  7 were  successful 
in  completing  the  three  steps.  The  timing  and  error  data  for  the  269  participants  who  were 
successful  in  this  second  task  is  presented  in  Table  12. 


Participants 

Instructional 

Method 

Mean 

Time 

(seel 

Median 
Time  (sec) 

Mean 

Errors 

Median 

Errors 

Max 

Errors 

269 

All 

52.63 

46 

0.17 

0 

5 

90 

Poster 

53.89 

46 

0.19 

0 

5 

87 

Verbal 

49.56 

45 

0.10 

0 

2 

92 

Video 

54.29 

46 

0.21 

0 

4 

Table  12:  Error  Analysis  For  Reference  Task 


We  compared  the  timing  data  for  the  reference  task  in  Table  12  to  the  timing  data  for  the 
original  task  presented  in  Table  6.  The  tables  have  been  combined  for  convenience  in  Table 


13. 
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Method 

Original  Task 

Reference  Task 

Participants 

Mean 

Time  (sec) 

Median 
Time  (sec) 

Participants 

Mean  Time 
(sec) 

Median 

Time 

Poster 

52 

64.13 

58.50 

90 

53.89 

46.00 

Verbal 

85 

48.21 

45.00 

87 

49.56 

45.00 

Video 

85 

50.87 

46.00 

92 

54.29 

46.00 

Table  13:  Comparison  of  Medians  for  Original  and  Reference  Task 


To  complete  the  analysis  we  used  a Mann- Whitney  test. 

1 . For  the  poster,  the  medians  of  58.50  and  46.50  are  different  with  a p-value 

2.  For  the  verbal  condition,  the  medians  of  45.00  and  45.00  are  not  different, 

0.05. 

3.  For  the  video  condition,  the  medians  of  46.00  and  45.00  are  not  different, 

>0.05. 

For  this  operator  assisted  second  task  97%  of  the  participants  were  able  to  successfully 
complete  the  task  compared  to  78%  for  the  first  task.  Overall  those  in  the  poster  condition 
committed  fewer  errors  and  completed  the  task  faster  than  the  original  task. 


<0.01 
p-value  > 

p-value 


4.  DISCUSSION 


This  study  was  designed  to  examine  the  impact  of  instructional  mode  on  user  performance. 
In  particular  the  goals  were  to  determine: 

1.  How  the  instructional  mode  (poster,  video,  or  verbal)  affected  user  performance? 

2.  How  long  it  took  to  capture  a 10-print  image?  and 

3.  What  were  the  frequency  and  nature  of  the  errors  that  occurred  in  this  process? 

We  address  these  questions  with  respect  to  each  instructional  method. 

4.1  POSTER 

Participants  who  received  instructions  via  the  poster  had  the  most  difficulty  performing  the 
fingerprinting  task.  54%  of  these  participants  were  unable  to  successfully  complete  the 
fingerprinting  process,  those  who  did  took  longer  and  committed  more  errors.  The  reasons 
for  these  errors  varied.  Some  of  the  participants  stated  that  the  poster  offered  too  much 
information  and  did  not  inform  them  as  to  the  most  important  parts.  Therefore,  participants 
remembered  only  certain  pieces  of  information,  neglecting  others.  The  most  common  error 
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made  by  these  participants  was  removing  their  fingers  before  a capture  had  taken  place. 
During  the  follow-up  interviews  participants  explained  that  the  poster  either  failed  to  show 
the  importance  of  the  green  lights  or  failed  to  show  that  the  green  lights  were  flashing.  Some 
participants  noted  that  they  failed  to  see  the  lights  at  all.  Overall  participants  who  removed 
their  hand  too  quickly  stated  that  the  poster  failed  to  articulate  any  timing  information. 

Participants  also  did  not  seem  to  notice  from  the  poster  that  the  lights  on  the  fingerprinting 
device  conveyed  significant  information  in  the  fingerprint  capture  process.  Participants  who 
did  notice  the  light  were  perplexed  to  see  a blinking  light  on  the  device.  They  had  not 
gleaned  from  the  poster  illustration  that  a blinking  light  was  involved. 

An  important  consideration  with  the  poster  is  the  short  time  that  participants  took  to  study  it. 
Many  times  participants  would  only  glance  at  the  poster  and  then  continue  with  the  process. 
With  the  verbal  or  video  presentations,  participants  were  more  inclined  to  wait  until  the 
instructions  had  run  their  course.  Thus  they  spent  more  time  concentrating  on  the  instructions 
than  when  confronted  with  the  poster.  The  lack  of  a time  dimension  of  the  poster  instructions 
also  led  to  some  confusion  as  to  the  required  length  of  time  of  the  parts  of  the  fingerprinting 
process.  Many  participants  guessed  at  the  time  required  to  give  the  fingerprints. 

A number  of  participants  also  stated  that  in  the  poster  it  looked  as  if  the  hand  was  pressing  a 
button  with  the  thumb  before  placing  the  hand  on  the  scanner.  This  led  to  people  looking  for 
a button  on  the  device  and  then  experiencing  confusion  when  they  could  not  find  it. 

One  persistent  issue  with  regards  to  the  poster  instructions  was  that  participants  often 
expected  further  instructions  to  follow.  There  were  multiple  instances  where  the  participant 
would  view  the  poster,  approach  the  fingerprint  device,  and  then  ask  the  operator  for 
instructions.  Once  the  operator  informed  the  participant  that  he/she  was  not  in  a position  to 
answer  any  questions,  the  participant  would  ask  to  see  the  poster  again  or  simply  explain  that 
they  only  briefly  viewed  the  poster  as  they  expected  further  details. 

4.2  VIDEO 

Participants  who  viewed  the  video  instructions  generally  performed  well  in  the  fingerprinting 
task.  One  problem  identified  by  the  participants  was  with  the  timing  of  the  green  light 
turning  off.  People  stated  they  could  not  tell,  or  had  to  watch  more  than  once  to  determine,  if 
the  light  went  off  before  or  after  the  hand  was  removed.  Participants  seemed  split  in  their 
opinions  about  the  duration  of  the  video.  Some  people  stated  the  video  was  excessively  long 
and  that  the  critical  information  could  be  conveyed  in  a shorter  version.  Others  stated  that  the 
fact  the  video  was  in  real-time  helped  to  inform  them  as  to  how  long  to  keep  their  hands  on 
the  scanner.  This  information  was  derived  independently  from  their  realization  about  the 
purpose  of  the  lights,  as  some  people  actually  counted  the  seconds  as  the  video  played. 
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Another  problem  associated  with  the  video  instructions,  was  that  the  hand  in  the  video 
seemed  to  press  a button  with  the  thumb.  They  also  noted  that  it  would  have  been  helpful  to 
have  been  informed  at  the  beginning  of  the  video  that  there  were  three  steps  demonstrated  in 
the  video. 

4.3  VERBAL 

Participants  overwhelmingly  stated  that  the  verbal  instructions  clearly  described  the  task.  In 
general,  participants  who  received  the  verbal  instructions  fared  better  than  those  who  were 
instructed  by  the  poster.  Some  concerns  raised  by  participants  about  the  verbal  instructions 
were  that  they  seemed  “too  wordy”  or  that  they  were  overly  complicated. 

The  few  mistakes  made  following  the  verbal  instructions  could  be  attributed  to  the  lack  of  a 
frame  of  reference.  Since  all  participants  were  using  the  device  for  the  first  time,  the 
instructions  were  being  read  without  any  visual  reference,  therefore,  leaving  room  for 
interpretation.  The  instructions  inform  the  participant  to  place  his/her  finger  on  the  scanner. 
They  do  not  specify  where  on  the  scanner  to  place  the  fingers,  and  as  a result  some  people 
placed  their  fingers  on  the  plastic  casing  located  behind  the  platen.  During  the  follow-up 
interview  participants  stated  that  a lack  of  reference  in  regards  to  the  green  lights  also  created 
confusion  during  the  instructions.  Not  having  a sense  of  what  the  device  would  look  like 
until  they  approached  it  also  caused  participants  who  were  instructed  verbally  to  take  extra 
time  inspecting  the  device,  trying  to  relate  what  they  were  seeing  to  what  they  had  previously 
heard. 

5.  CONCLUSIONS  AND  FUTURE  WORK 


This  study  examined  the  impact  of  instructional  mode  on  user  performance,  hi  particular  we 
examined  the  impact  of  poster,  video,  and  verbal  instructions  on  the  time  required  to  capture 
a 10-print  slap  as  well  as  the  number  of  errors  incurred  during  the  process  For  this  study  a 
10-print  slap  capture  includes  a right  slap,  left  slap  and  simultaneous  thumbs  images.  This 
study  was  performed  in  a controlled  environment.  This  controlled  environment  did  not 
attempt  to  simulate  a point  of  entry  environment  or  the  stress  associated  with  traveling  and 
encountering  the  fingerprint  process. 

From  this  study  we  observed  four  main  results.  First,  participants  who  received  instructions 
via  the  poster  had  the  most  difficulty  performing  the  fingerprinting  task.  Only  56%  of  these 
participants  were  able  to  successfully  complete  the  fingerprinting  process.  Participants  who 
received  instructions  by  poster  and  successfully  completed  the  fingerprinting  capture  process 
took  significantly  longer  and  committed  significantly  more  errors  than  participants  receiving 
video  or  verbal  instructions.  Although  the  poster  or  placard  is  a cost  effective  method  of 
providing  information  for  the  2 -print  process,  this  study  illustrates  the  challenges  of 
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depicting  a dynamic  process  in  a static  medium  for  a more  complex  10-print  process.  We 
took  great  care  in  developing  the  poster  for  this  study.  Numerous  versions  of  the  poster  were 
developed  as  we  iteratively  tested  each  version  with  small  test  groups  before  finalizing  the 
design. [NIST6]  However,  additional  study  is  required  to  develop  a family  of  static  symbols 
that  can  be  used  selectively  to  depict  dynamic  processes  if  poster  instructions  are  to  be  used 
at  all. 

Second,  participants  who  received  the  verbal  and  video  instructions  performed  equally  well. 
While  the  verbal  method  may  have  been  equally  as  effective  as  the  video  in  providing 
optimal  capture  time  and  minimal  error  count,  in  this  particular  context,  in  an  operational 
setting  it  may  not  always  be  possible  to  depend  on  verbal  instructions  due  to  language 
barriers.  Moreover,  the  verbal  instructions  must  be  delivered  to  each  participant 
individually,  in  this  study  the  verbal  instructions  took  approximately  45  seconds.  With  this 
in  mind,  the  video  instructions  may  be  more  conducive  to  the  operational  environment. 
Video  instructions  can  be  presented  to  all  the  participants  at  once  as  they  wait  in  line. 

Video  instructions  may  be  the  optimal  method  of  instructional  delivery  providing  for 
efficiency  by  adding  no  additional  time  for  the  operator  during  the  collection  process  and  still 
providing  optimum  capture  times  and  low-error  incidence. 

Third,  operators  are  critical  to  the  acquisition  process.  It  appears  that  operators  are  able  to 
assist  individuals  to  overcome  the  deficits  of  the  instructional  materials.  Without  the 
assistance  of  an  operator  45%  of  the  participants  who  received  the  poster  were  not  able  to 
complete  the  fingerprinting  process  correctly.  Overall,  only  78%  were  able  to  successfully 
complete  the  collection  process  without  assistance  compared  to  98%  with  operator 
assistance.  In  addition,  participants  for  the  poster  condition  committed  fewer  errors  and  took 
less  time  when  assisted  by  an  operator. 

Finally  we  measured  the  time  required  to  capture  a 10-print  sequence — in  this  case  a right 
slap,  left  slap  and  simultaneous  thumbs.  With  our  biometric  system  configuration  and  a five 
second  capture  time,  the  minimum  time  to  successfully  complete  a capture  sequence  of  the 
three  images  is  approximately  30  seconds.  This  includes  approximately  21  seconds  for 
image  capture  and  9 seconds  of  overhead  for  initialization  and  other  operational  chores.  On 
average  without  operator  assistance  the  collection  process  ranged  from  48  to  64  seconds 
while  the  medians  ranged  from  45  to  59  seconds,  and  with  operator  assistance  the  collection 
process  ranged  from  50  to  54  seconds  and  the  medians  ranged  from  45  to  46  seconds. 
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APPENDIX  A:  DEMOGRAPHIC  QUESTIONNAIRE 
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Demographic  Questionnaire 


1.  Age 

□ 18  to  24 

□ 25  to  34 

□ 35  to  44 

□ 45  to  54 

□ 55  to  64 

□ 65  + 


2.  Gender 

□ Male 

□ Female 


3.  Handedness  4.  Height 

O Right  Handed  I 1 Feet  1 1 Inches 

□ Left  Handed 

□ Ambidextrous 


5.  Country  of  Origin 


6.  Profession 


7.  Have  you  ever  had  your  fingerprints  captured  before? 


□ No 

If  you  answered  Yes,  check  all  that 

□ Yes  ■< 

apply: 

□ Fingerprinted  electronically 

□ Fingerprinted  with  ink/paper 

8.  Before  you  took  part  in  the  trial  how  concerned  were  you  about  having  your 
fingerprints  recorded? 

□ Very  concerned 

□ Fairly  concerned 

□ Not  very  concerned 

□ Not  at  all  concerned 

□ Don’t  know 
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APPENDIX  B:  POST  TEST  QUESTIONNAIRE 
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Task  Evaluation  Questionnaire 


1.  Was  the  level  of  intrusiveness  you  experienced  during  the  fingerprint  image 
recording  more  or  less  than  you  expected? 

□ Much  more  than  expected 

□ A little  more  than  expected 

□ About  the  same  as  expected 

□ A little  less  than  expected 

□ A lot  less  than  expected 

□ Don’t  know 


2.  How  did  the  time  it  took  to  have  your  fingerprint  image  recorded  compare  with  what 
you  expected? 

□ Much  more  than  expected 

□ A little  more  than  expected 

□ About  the  same  as  expected 

□ A little  less  than  expected 

□ A lot  less  than  expected 

□ Don’t  know 


3.  How  would  you  rate  the  difficulty  in  positioning  yourself  so  that  your  fingerprint 
image  could  be  recorded? 

□ Much  more  than  expected 

□ A little  more  than  expected 

□ About  the  same  as  expected 

□ A little  less  than  expected 

□ A lot  less  than  expected 

□ Don't  know 


4.  Overall  how  did  you  feel  about  the  experience  of  having  your  fingerprints  taken? 


□ Much  better  than  expected 

□ A little  better  than  expected 
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□ About  the  same  as  expected 

□ A little  worse  than  expected  <4- 

□ A lot  worse  than  expected  + 

□ Don’t  know 


If  you  answered  worse,  can  you  please  elaborate? 


5.  If  fingerprint  images  were  adopted  as  a means  of  establishing  identity  for  passport 
purposes  would  you  be  in  favor  or  opposed  to  providing  an  image  of  your  fingerprints? 

□ Strongly  in  favor 

□ In  favor 

□ Neither  in  favor  nor  opposed 

□ Opposed 

□ Strongly  opposed 

□ Don't  know 


6.  Rate  the  following  questions  on  a 5 point  scale  with  1 being  least  favored  and  5 most. 


Question 

© 

5 

4 

© 

3 

2 

© 

1 

The  information  provided  was  effective  in 
helping  me  complete  the  fingerprint  process. 

O 

O 

O 

O 

O 

The  information  provided  clearly  described 
the  fingerprint  process. 

O 

O 

O 

O 

O 

The  information  provided  was  confusing. 

O 

O 

O 

O 

O 

It  was  clear  which  hand  was  to  be  scanned 
first. 

O 

O 

O 

O 

O 

The  order  of  fingerprint  capture  was  clear. 

O 

O 

O 

O 

O 

It  was  clear  how  to  position  your  hand  on  the 
sensor. 

O 

O 

O 

O 

O 

It  was  clear  when  to  remove  your  hand. 

O 

O 

O 

O 

O 
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7.  What  was  the  most  confusing  part  of  the  process? 


9.  Would  information  in  another  form  be  more  helpful  (rank  order) 
(write  in  1-5) 

Pamphlet 
Poster 
Video 
Live  demo 
Other  (specify) 


10.  What  additional  information  would  you  recommend  to  improve  the  fingerprint  capture 
process? 


NOTE:  This  survey  contains  collection  of  information  requirements  subject  to  the 
Paperwork  Reduction  Act.  Notwithstanding  any  other  provision  of  the  law,  no  person  is 
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required  to  respond  to,  nor  shall  any  person  be  subject  to  a penalty  for  failure  to  comply  with, 
a collection  of  information  subject  to  the  requirements  of  the  Paperwork  Reduction  Act, 
unless  that  collection  of  information  displays  a currently  valid  OMB  control  number.  The 
estimated  response  time  for  this  survey  is  15  minutes.  The  response  time  includes  the  time 
for  reviewing  instructions,  searching  existing  data  sources,  gathering  and  maintaining  the 
data  needed,  and  completing  and  reviewing  the  collection  of  information." 

OMB  no  0693-0043,  expiration  07/31/09 
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APPENDIX  C:  INSTRUCTIONAL  POSTERS 
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Figure  21:  Instructional  Poster  (Right  Hand  Start) 
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2. 


Figure  22:  Instructional  Poster  (Left  Hand  Start) 
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APPENDIX  D:  VERBAL  INSTRUCTIONS 
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The  following  test  script  was  verbally  presented  to  each  participant  in  our  study  by  the  test 
system  operator.  Portions  of  this  script  that  have  been  enclosed  in  brackets  (“[“  and  “]”)  and 
are  meant  as  special  instructions  for  the  operator.  These  special  instructions  guide  the 
operator  to  tailor  instructions  for  the  specific  participant  or  to  have  the  operator  perform 
certain  actions. 


[Welcome  the  participant] 

“Thank  you  for  taking  the  time  to  participate  in  our  study  today. 

My  name  is  [name]  and  we’re  interested  in  researching  how  long  it  takes  for  all  ten 
fingerprints  to  be  given,  using  an  electronic  digital  fingerprint  scanner.  We  are  also 
interested  in  the  process  of  taking  Iris  and  Face  images” 

“Today’s  session  will  last  approximately  30  minutes.  If  you  want  to  stop  at  any  time  just 
say  so.” 

DESCRIBE  EQUIPMENT 

“In  our  study,  we’ll  be  using  a digital  fingerprint  scanner  connected  to  a computer  that 
detects  and  analyzes  fingerprint  images  from  the  scanner. 

You  will  be  asked  to  use  this  scanner  to  give  all  ten  of  your  fingerprints.” 

“There  will  be  people  observing  the  fingerprinting  process  so  we  can  keep  track  of  how 
long  the  different  parts  of  the  process  take.” 

“Also,  parts  of  the  process  will  be  video  taped  so  that  we  can  analyze  it  in  more  detail. 
However,  we  are  not  recording  anything  of  you  that  can  be  use  for  personal  identification 
purposes. 

Consent  Form: 

Demographic  Form: 

EXPLANATION  OF  TASKS 

“You  will  be  presented  with  the  instructions  on  how  to  use  the  scanner  to  give  your 
fingerprints.  After  you  feel  that  you  understand  the  instructions,  proceed  to  the  operator 
with  your  papers.” 

“Do  you  have  any  questions  before  we  begin?” 
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If  yes:  (answer  questions  or  explain  again.  No  mention  of  the  three  instructional 
techniques  should  be  made  to  the  participant  or  anything  specific  about  the 
scanning  process). 


If  no:  take  participant  into  instructional  area 

(Participant’s  actions  are  recorded  via  Noldus) 
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Right  Hand  first  instructions: 

Instructions  to  Participant: 

Leaving  a 10-print  image  is  simple  and  consists  of  three  steps. 

Step  #1 : When  the  right  most  light  starts  flashing  green;  place  your  right  four  fingers 
(demonstrate  what  four  fingers)  on  the  scanner  and  press  down.  Hold  your  fingers  in 
place  until  the  green  light  stops  flashing 


Step  #2:  When  the  left  most  light  starts  flashing  green;  place  your  left  four  fingers 
(demonstrate  what  four  fingers)  on  the  scanner  and  press  down.  Hold  your  fingers  in 
place  until  the  green  light  stops  flashing 


Task  #3:  When  the  center  light  starts  flashing  green;  place  your  2 thumbs  (demonstrate 
thumbs  side  by  side)  on  the  scanner  and  press  down.  Hold  your  fingers  in  place  until  the 
green  light  stops  flashing. 


Would  you  like  me  to  repeat  the  instructions? 
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Left  Hand  first  instructions: 


Leaving  a 10-print  image  is  simple  and  consists  of  three  steps. 

Step  #1:  When  the  left  most  light  starts  flashing  green;  place  your  left  four  fingers 
(demonstrate  what  four  fingers)  on  the  scanner  and  press  down.  Hold  your  fingers  in 
place  until  the  green  light  stops  flashing. 

Step  #2:  When  the  right  most  light  starts  flashing  green;  place  your  right  four  fingers 
(demonstrate  what  four  fingers)  on  the  scanner  and  press  down.  Hold  your  fingers  in 
place  until  the  green  light  stops  flashing. 

Task  #3:  When  the  center  light  starts  flashing  green;  place  your  2 thumbs  (demonstrate 
thumbs  side  by  side)  on  the  scanner  and  press  down.  Hold  your  fingers  in  place  until  the 
green  light  stops  flashing. 

Would  you  like  me  to  repeat  the  instructions? 
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PRESENT  1 OF  THREE  INSTRUCTIONAL  TECHNIQUES 


When  the  participant  understands  the  instructions  they  go  into  scanning  area. 

OPERATOR: 

1 . Retrieves  papers  from  participant. 

2.  Presses  MBARK  Start  Capture  (MBARK  should  be  set  up  for  the  participant 
previous  to  this  point) 

3.  Tells  Participant  “you  may  begin” 

Capture  process  occurs  ... 

Questions  that  may  occur: 

1 . During  the  instructional  period,  participant  may  ask  “Can  I begin  now?”  respond 
with  “If  you  are  ready,  please  continue”. 

2.  Generally,  if  the  participant  asks  any  question  about  the  directions  for  giving  the 
prints  (e.g.,  “how  long  should  I leave  my  hand?”),  respond  with  “You  should 
perform  as  the  instructions  have  indicated”. 

3.  “What  does  the  (...)  light  mean?”  Don’t  explain  “I’d  be  happy  to  explain  at  the 
end  of  the  test. 

4.  “Is  the  scanner  clean?”  “Yes,  we  clean  it  between  each  participant” 

5.  “May  I see  the  instructions  again?”  “You  may  see  the  instructions  after  the  last 
scan” 

6.  “(Why  is  the  scanner)/(Should  the  scanner  be)  warm?”  “This  is  normal  system 
behavior.” 

Actions  that  may  occur: 

The  general  rule  is  to  only  prompt  and  make  corrections  during  the  session  if  the 
participant  communicates  that  a mistake  was  made. 

Participant  attempts  wrong  task,  but  does  not  notice 

If  the  participant  uses  the  incorrect  hand,  but  does  not  notice,  then  make  the 
appropriate  corrections  after  the  participant  is  finished  presenting  their  fingerprints, 
but  before  the  “End  Session”  button  is  clicked. 

Participant  notices  they  have  attempted  the  wrong  task 

If  the  participant  uses  the  incorrect  hand  and  does  notice,  then  the  operator  should 
take  the  appropriate  action  (such  as  canceling  a capture  if  one  was  already  in  process) 
and  ask  the  participant  to  “Please  wait  while  I make  a correction.” 
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Participant  thinks  they  have  used  the  incorrect  hand,  but  hasn’t 

Tell  the  participant  to  continue  with  the  rest  of  the  tasks  and  that  they  can  try  again 
after  completion  of  the  first  set  of  tasks.  Operator  starts  a new  session  after  completion. 

Participant  leaves  their  hand  on  too  long 

If  the  participant  leaves  their  hand  on  the  scanner  for  too  long  (and  the  red  LED  is 
flashing)  the  operator  should  say  “You  may  remove  your  hand  now.” 

Participant  lifts  their  hand  too  soon  and  goes  on  to  next  task  without  waiting  for 
indication. 

If  the  participant  lifts  their  hand(s)  too  soon,  the  operator  should  make  no  verbal 
correction.  If  the  participant  attempts  to  start  the  next  task,  before  the  system  detects 
that  they  have  lifted  their  hand(s)  too  soon,  then 
Allow  participant  to  continue. 

Participant  lifts  their  hand  too  soon  then  replaces  the  same  hand. 

If  the  participant  lifts  their  hand(s)  too  soon,  then  replaces  hand  before  the  critical  period. 
Mark  it  as  a Noldus  error  but  the  operator  does  not  reject  the  capture. 

Participant  assumes  they  are  done 

If  the  participant  assumes  that  they  are  (prematurely)  finished  with  the  session  then 
the  operator  should  stop  the  capture  and  ask  the  participant  if  they  would  like  to  try 
again  (it  is  important  that  the  participant  does  not  feel  they  did  something  wrong,  just 
we  would  like  to  collect  another  set  of  prints).  Operator  starts  new  session. 

Participant  asks  if  they  are  done 

If  the  participant  asks  if  they  are  finished  with  the  session,  and  they  are  not , then 
“If  you  believe  that  you  have  successfully  given  the  prints,  then  you  are  done.” 

If  the  participant  assumes  that  they  are  (prematurely)  finished  with  the  session  then 
the  operator  should  stop  the  capture  and  ask  the  participant  if  they  would  like  to  try 
again  (it  is  important  that  the  participant  does  not  feel  they  did  something  wrong,  just 
we  would  like  to  collect  another  set  of  prints).  Operator  stars  new  session. 

Timeouts 

Every  capture  has  an  associated  timeout — the  time  during  which  a successful  capture 
must  take  place,  or  the  system  will  consider  the  attempt  as  failed.  However,  because  a 
successful  capture  requires  a span  of  time,  then  there  is  some  critical  time  during 
which  if  the  participant  starts,  a failure  is  inevitable.  For  example,  if  the  system  is 
configured  to  capture  fingerprints  for  five  seconds,  and  timeout  in  30  sections,  a 
subject  that  starts  the  task  28  seconds  into  the  process  cannot  give  a successful 
capture  without  timing  out. 
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If  a capture  process  times  out  because  the  participant  does  not  attempt  the  task  during 
the  allotted  time,  or  if  the  system  can  not  detect  that  a hand  was  placed  on  the 
scanner,  then  the  operator  should  make  no  verbal  correction  and  allow  the  system  to 
continue  to  operate  along  its  workflow.  If  the  operator  observes  that  the  subject 
places  their  hand  on  the  sensor  during  the  critical  time  then  the  operator  should 
prompt  the  participant  to  “Please  try  again.” 

4.  Ends  MBARK  session  (Noldus  tracking  stops) 

QUESTIONNAIRE 

“Thank  you  for  your  participation.  That  concludes  the  test.” 

“We  have  a brief  questionnaire  that  we  would  like  you  to  complete.” 

Iris  and  Face  Scan 

TBD 


CONCLUSION 

“Thanks  again  for  taking  the  time  to  participate  in  our  study.” 

“Do  you  have  any  comments  or  questions  about  today’s  test  session?” 
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APPENDIX  E:  NOLDUS  EVENTS 
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Short  Code 

Code  Description 

Long  Code  & Button  Title  in  Noldus  UI 

ENT 

Subj  ect  enters  room 

RmEntr 

REA 

Subject  crosses  ready  line 

RdyLine 

DOC 

Subject  hands  paperwork  to  operator 

DocsGivn 

ORE 

Operator  asks  if  subject  is  ready 

OpAskReady 

SRE 

Subject  indicates  he/she  is  ready 

SubjRdy 

HLO 

Left  hand  placed  on  scanner 

Left  On 

HOF 

Subject  removes  hand(s)  from  scanner 

Hnds  Off 

HRO 

Right  hand  placed  on  scanner 

Right  On 

THU 

Subject  places  both  thumbs  on  scanner 

Thmbs  On 

END 

All  three  prints  captured 

End  Task 

ER1 

Wrong  hand  placed  on  scanner 

E-WrongHand 

ER2 

Hand  lifted  too  soon 

E-EarlyLift 

ER3 

General  error 

E-General 

QUE 

Subject  asks  operator  question 

Question 

REV 

Review  tape  at/around  this  time  index 

ReviewTape 

RLO 

Red  Light  On 

M-ROn 

GLO 

Green  Light  On 

M-GOn 

ALO 

All  Lights  Off 

M-LOf 

CSL 

Capture  Start-Left  Slap 

M-CapSL 

CSR 

Capture  Start-Right  Slap 

M-CapSR 

CST 

Capture  Start-Thumbs 

M-CapST 

CFR 

Capture  Frame 

M-CapF 

CEN 

Capture  End 

M-CapE 

CSS 

Capture  Start  Session 

M-CapSS 

CES 

Capture  End  Session 

M-CapES 

CHO 

Capture  Hand(s)  Off 

- -M-CapHO 

CCA 

Capture  Cancel 

M-CapC 

CPR 

Premature  Lift 

M-CapP 

TCO 

Task  correction 

M-TskC 

INI 

Initialization 

M-Int 

CTO 

Capture  Time-out 

— - M-CapT 

OST 

Operator  skips  task 

M-CapSK 

MGE 

Generic  MBARK  event 

— M-Gen 

Table  14:  Noldus  Event  Codes 
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